Authentication using facial image comparison

ABSTRACT

Computer systems and methods are provided for transmitting authorization information to an image capturing device. A computing system receives, from an image capturing device, captured image data that includes a first facial image and an image of a document that includes a second facial image. The first facial image and the second facial image are included a single image frame. The captured image data is analyzed to determine a first portion of the captured image data that corresponds to the first facial image and a second portion of the captured image data that corresponds to the second facial image. The first portion of the captured image data is compared with the second portion of the captured image data. In accordance with a determination that the first facial image and the second facial image meet matching criteria, authorization information is transmitted to the image capturing device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of Ser. No. 16/795,502,filed Feb. 19, 2020, which is a continuation application of Ser. No.15/673,324, filed Aug. 9, 2017, now U.S. Pat. No. 10,606,993. Thisapplication is related to U.S. patent application Ser. No. 13/968,164,filed Aug. 15, 2013, entitled “Image Processing for Credit CardValidation,” and U.S. Provisional Patent Application No. 61/683,623,filed Aug. 15, 2012, entitled “Image Processing for Credit CardValidation.” All of these applications are incorporated by reference intheir entirety.

TECHNICAL FIELD

This application relates generally to user authentication, and moreparticularly, to using facial image data for user authentication.

BACKGROUND

User authentication is often performed prior to granting user access toa system. Typically, user authentication involves accessing previouslystored user information, such as user identification information and/oruser biometric data, and comparing the previously stored userinformation with information that a user provides in connection with anaccess request. Systems that perform user authentication store userinformation in a data storage device. Prior to requesting authorization,users enroll with the system by providing user information to be stored.

SUMMARY

Some authentication systems described herein perform authenticationusing a captured image that includes a person's face and a document thatincludes a previously captured image of the person's face. For example,while the person is holding an identification document, such as adriver's license, that includes a previously captured photograph of theperson's face, an image is captured such that the person's face and theidentification document are both visible in the image. Image analysis isperformed on the image to determine whether the person's face in theimage matches the facial image on the identification document. If theimage analysis determines that there is a match, authorization isgranted. In this way, a device is enabled to perform authenticationusing a received image without relying on access to user informationobtained and stored prior to receiving the image.

In some embodiments, a method is performed at a server system includingone or more processors and memory storing one or more programs forexecution by the one or more processors. The method includes receiving,from an image capturing device, captured image data that includes: afirst facial image and an image of a document that includes a secondfacial image. The first facial image and the image of the document thatincludes the second facial image are included a single image frame. Thecaptured image data is analyzed to determine a first portion of thecaptured image data that corresponds to the first facial image and asecond portion of the captured image data that corresponds to the secondfacial image. The method additionally includes determining whether thefirst facial image and the second facial image meet matching criteria bycomparing the first portion of the captured image data with the secondportion of the captured image data. In accordance with a determinationthat the first facial image and the second facial image meet thematching criteria, authorization information is transmitted to the imagecapturing device.

In some embodiments, a system includes one or more processors, memory,and one or more programs. The one or more programs are stored in thememory and configured to be executed by the one or more processors. Theone or more programs include instructions for receiving, from an imagecapturing device, captured image data that includes a first facial imageand an image of the document that includes a second facial image. Thefirst facial image and the image of the document that includes thesecond facial image are included a single image frame. The one or moreprograms additionally include instructions for analyzing the capturedimage data to determine a first portion of the captured image data thatcorresponds to the first facial image and a second portion of thecaptured image data that corresponds to the second facial image. The oneor more programs additionally include instructions for determiningwhether the first facial image and the second facial image meet matchingcriteria by comparing the first portion of the captured image data withthe second portion of the captured image data. The one or more programsadditionally include instructions for, in accordance with adetermination that the first facial image and the second facial imagemeet the matching criteria, transmitting authorization information tothe image capturing device.

In some embodiments, a computer readable storage medium stores one ormore programs. The one or more programs comprise instructions, whichwhen executed, cause a device to receive, from an image capturingdevice, captured image data that includes a first facial image and animage of the document that includes a second facial image. The firstfacial image and the image of the document that includes the secondfacial image are included a single image frame. The one or more programsadditionally cause the device to analyze the captured image data todetermine a first portion of the captured image data that corresponds tothe first facial image and a second portion of the captured image datathat corresponds to the second facial image. The one or more programsadditionally cause the device to determine whether the first facialimage and the second facial image meet matching criteria by comparingthe first portion of the captured image data with the second portion ofthe captured image data. The one or more programs additionally cause thedevice to, in accordance with a determination that the first facialimage and the second facial image meet the matching criteria, transmitauthorization information to the image capturing device.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the present disclosure can be understood in greater detail,features of various embodiments are illustrated in the appendeddrawings. The appended drawings, however, merely illustrate pertinentfeatures of the present disclosure and are therefore not limiting.

FIG. 1 is a system diagram of a computing system and its context, inaccordance with some embodiments.

FIG. 2 is a system diagram of an image capturing device, in accordancewith some embodiments.

FIG. 3 illustrates a document that includes a facial image, inaccordance with some embodiments.

FIGS. 4 and 5 illustrate image capture environments, in accordance withsome embodiments.

FIGS. 6A, 6B, 7A, and 7B illustrate captured images that include a firstfacial image and an image of a document that includes a second facialimage, in accordance with some embodiments.

FIG. 8A illustrates a first state of a user interface that displays amoving target for liveness verification, in accordance with someembodiments.

FIG. 8B illustrates an image that is captured while the moving target ofFIG. 8A is displayed, in accordance with some embodiments.

FIG. 9A illustrates a second state of a user interface that displays amoving target for liveness verification, in accordance with someembodiments.

FIG. 9B illustrates an image that is captured while the moving target ofFIG. 9A is displayed, in accordance with some embodiments.

FIGS. 10A-10B illustrate movement of an eye relative to movement of afacial image while the moving target of FIGS. 8A and 9A is displayed, inaccordance with some embodiments.

FIG. 11A illustrates a first state of a user interface that displayslanguage content for liveness verification, in accordance with someembodiments.

FIG. 11B illustrates an image that is captured while the languagecontent of FIG. 11A displayed, in accordance with some embodiments.

FIG. 12A illustrates a second state of a user interface that displayslanguage content for liveness verification, in accordance with someembodiments.

FIG. 12B illustrates an image that is captured while the languagecontent of FIG. 12A displayed, in accordance with some embodiments.

FIGS. 13A-13H are flow diagrams illustrating a method for authenticatinga user using facial image comparison, in accordance with someembodiments.

In accordance with common practice, some of the drawings may not depictall of the components of a given system, method, or device. Finally,like reference numerals denote like features throughout thespecification and figures.

DETAILED DESCRIPTION

Numerous details are described herein in order to provide a thoroughunderstanding of the example embodiments illustrated in the accompanyingdrawings. However, some embodiments may be practiced without many of thespecific details, and the scope of the claims is only limited by thosefeatures and aspects specifically recited in the claims. Furthermore,well-known processes, components, and materials have not been describedin exhaustive detail so as not to unnecessarily obscure pertinentaspects of the embodiments described herein.

FIG. 1 is a system diagram of a computing system 100, in accordance withsome embodiments. The computing system 100 is, for example, a servercomputer, a desktop computer, or a laptop computer. The computing system100 typically includes a memory 102, one or more processor(s) 120, apower supply 122, an input/output (I/O) subsystem 124, and acommunication bus 126 for interconnecting these components.

The processor(s) 120 execute modules, programs, and/or instructionsstored in the memory 102 and thereby perform processing operations.

In some embodiments, the memory 102 stores one or more programs (e.g.,sets of instructions) and/or data structures, collectively referred toas “modules” herein. In some embodiments, the memory 102, or thenon-transitory computer readable storage medium of the memory 102 storesthe following programs, modules, and data structures, or a subset orsuperset thereof:

-   -   an operating system 104;    -   an image analysis module 106;    -   a user authentication module 108, which stores information such        as captured image data 110, extracted first facial image data        112, and/or extracted second facial image data 114 (e.g.,        extracted by the image analysis module 106 from the captured        image data 110), user identification information 116 (e.g., user        name, user password, user residential information, user phone        number, user date of birth, and/or user e-mail), and user        biometric information 118 (e.g., facial data, fingerprint data,        retinal data, hand image data, and/or gait data); and    -   liveness analysis module 119, which stores information for        displaying a moving target liveness user interface 800 and/or a        language content liveness user interface 1100, generates audio        output including facial movement instructions and/or language        content, stores verification data (e.g., facial feature position        data, audio print data that corresponds to language content        output, and/or facial image data that corresponds to language        content output), and/or uses an audio analysis module to perform        audio analysis.

The above identified modules (e.g., data structures, and/or programsincluding sets of instructions) need not be implemented as separatesoftware programs, procedures, or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousembodiments. In some embodiments, the memory 102 stores a subset of themodules identified above. In some embodiments, a remote authenticationdatabase 154 and/or a local authentication database 130 store one ormore modules identified above. Furthermore, the memory 102 may storeadditional modules not described above. In some embodiments, the modulesstored in the memory 102, or a non-transitory computer readable storagemedium of the memory 102, provide instructions for implementingrespective operations in the methods described below. In someembodiments, some or all of these modules may be implemented withspecialized hardware circuits that subsume part or all of the modulefunctionality. One or more of the above identified elements may beexecuted by one or more of the processor(s) 120. In some embodiments,one or more of the modules described with regard to the memory 102 isimplemented in the memory 202 of an image capturing device 200 (FIG. 2)and executed by the processor(s) 220 of the image capturing device 200.

In some embodiments, the I/O subsystem 124 communicatively couples thecomputing system 100 to one or more local devices, such as a biometricinput device 128 and/or a local authentication database 130, via a wiredand/or wireless connection. In some embodiments, the I/O subsystem 124communicatively couples the computing system 100 to one or more remotedevices, such as a remote authentication database 154, a first imagecapturing device 200 a, and/or a second image capturing device 200 b,via a first communications network 150, a second communications network152, and/or via a wired and/or wireless connection. In some embodiments,the first communications network 150 is the Internet. In someembodiments, the first communication network 150 is a first financialnetwork and the second communication network 152 is a second financialnetwork.

In some embodiments, a biometric input device 128 (e.g., a fingerprintscanner, a retinal scanner, and/or a camera) is communicatively coupledto the computing system 100. For example, the computing system 100 islocated in or near to an authentication kiosk, or is communicativelycoupled to an authentication kiosk that includes the biometric inputdevice 128.

The communication bus 126 optionally includes circuitry (sometimescalled a chipset) that interconnects and controls communications betweensystem components.

FIG. 2 is a system diagram of an image capturing device 200 (e.g., thefirst or second image capturing devices 200 a or 200 b), in accordancewith some embodiments. The image capturing device 200 typically includesa memory 202, a camera 218, one or more processor(s) 220, a power supply224, an input/output (I/O) subsystem 226, and a communication bus 228for interconnecting these components. The image capturing device 200 is,for example, a mobile phone, a tablet, a digital camera, a laptopcomputer or other computing device, or a kiosk.

The processor(s) 220 execute modules, programs, and/or instructionsstored in the memory 202 and thereby perform processing operations.

In some embodiments, the memory 202 stores one or more programs (e.g.,sets of instructions) and/or data structures, collectively referred toas “modules” herein. In some embodiments, the memory 202, or thenon-transitory computer readable storage medium of the memory 202 storesthe following programs, modules, and data structures, or a subset orsuperset thereof:

-   -   an operating system 204;    -   captured image data 206 (e.g., image data captured by the camera        218, such as video and/or still images); and    -   user identification information 208 (e.g., user name, user        password, user residential information, user phone number, user        date of birth, and/or user e-mail address).

The above identified modules (e.g., data structures, and/or programsincluding sets of instructions) need not be implemented as separatesoftware programs, procedures, or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousembodiments. In some embodiments, the memory 202 stores a subset of themodules identified above. In some embodiments, the camera 218 stores oneor more modules identified above (e.g., captured image data 206).Furthermore, the memory 202 may store additional modules not describedabove. In some embodiments, the modules stored in the memory 202, or anon-transitory computer readable storage medium of the memory 202,provide instructions for implementing respective operations in themethods described below. In some embodiments, some or all of thesemodules may be implemented with specialized hardware circuits thatsubsume part or all of the module functionality. One or more of theabove identified elements may be executed by one or more of theprocessor(s) 220. In some embodiments, one or more of the modulesdescribed with regard to the memory 202 is implemented in the memory 102of the computing system 100 and executed by processor(s) 120 of thecomputing system 100.

The camera 218 captures still images, sequences of images, and/or video.In some embodiments, the camera 218 is a digital camera that includes animage sensor and one or more optical devices. The image sensor is, forexample, a charge-coupled device or other pixel sensor that detectslight. In some embodiments, one or more optical devices are movablerelative to the image sensor by an imaging device actuator. The one ormore optical devices affect the focus of light that arrives at the imagesensor and/or an image zoom property.

In some embodiments, the image capturing device 200 includes a camera218 (e.g., the camera 218 is located within a housing of the imagecapturing device 200). In some embodiments, the camera 218 is aperipheral device that captures images and sends captured image data 206to the I/O subsystem 226 of the image capturing device 200 via a wiredand/or wireless communication connection.

In some embodiments, the I/O subsystem 226 communicatively couples imagecapturing device 200 to one or more remote devices, such as a computingsystem 100, via a first communication network 150 and/or a secondcommunication network 152.

In some embodiments, a user input device 230 and/or an output device 232are integrated with the image capturing device 200 (e.g., as atouchscreen display). In some embodiments, a user input device 230and/or an output device 232 are peripheral devices communicativelyconnected to an image capturing device 200. In some embodiments, a userinput device 230 includes a microphone, a keyboard, and/or a pointerdevice such as a mouse, a touchpad, a touchscreen, and/or a stylus. Insome embodiments, the output device 232 includes a display (e.g., atouchscreen display that includes input device 230) and/or a speaker.

The communication bus 228 optionally includes circuitry (sometimescalled a chipset) that interconnects and controls communications betweensystem components.

In some embodiments, one or more user input devices and/or outputdevices (not shown), such as a display, touchscreen display, speaker,microphone, keypad, pointer control, zoom adjustment control, focusadjustment control, and/or exposure level adjustment control, areintegrated with the device 200.

FIG. 3 illustrates a document 300 that includes a facial image 302, inaccordance with some embodiments. The document 300 is, for example, anidentification card, a driver's license, a passport, a financialinstrument (e.g., credit card or debit card), a facility access card,and/or a photograph.

In some embodiments, the document 300 includes facial image location cueinformation (e.g., the concentric rectangles indicated at 304). Facialimage location cue information 304 is a visual indication on thedocument 300 of a location of the facial image 302 within the document300. For example, the concentric rectangles 304 that surround facialimage 302 provide a cue to indicate the location of the facial image 302within the document 300. In some embodiments, facial image location cueinformation includes one or more marks and/or pointers. For example,facial image location cue information indicates a facial image area thatis smaller than the full area of the document 300 and that includes thefacial image 302, such as a perimeter that indicates boundaries of thefacial image 302 or otherwise surrounds a facial image 302. In someembodiments, a facial image location cue is a background surrounding afacial image 302 (e.g., a background that has a predefined color and/orpattern). In some embodiments, a facial image location cue includes amaterial and/or texture of the facial image area of the document 300that is different from a material and/or texture of the remainder of thedocument 300.

FIG. 4 illustrates an image capture environment 400, in accordance withsome embodiments. In FIG. 4, a person 402 is presenting a document 300(e.g., the person's driver's license) in one hand and an image capturingdevice 200 a (e.g., a mobile device) in the other hand. The document 300includes a previously captured photograph of a facial image 302 of theperson 402. The image capturing device 200 a is used to capture an imageof the face of person 402 and of the document 300 that includes thefacial image 302 of the person 402. In some embodiments, the camera 218is a front-facing camera of the image capturing device 200 a, allowingthe person 402 to adjust imaging properties of the camera 218 (e.g., aposition and/or zoom level of the camera 218) while viewing the outputof the camera 218 on a display (e.g., an output device 232) of the imagecapturing device 200 a to ensure that the document 300 and the face ofthe person 402 are both visible in an image frame captured by the camera218.

FIG. 5 illustrates an image capture environment 500, in accordance withsome embodiments. In FIG. 5, the image capturing device 200 b is a kiosk(or a component of a kiosk). The kiosk 200 b is, for example, a securitykiosk (e.g., for gaining entrance to an entertainment venue, an office,and/or a travel destination) or a commercial kiosk (e.g., a registrationand/or check-out device for a commercial establishment such as a storeor hotel). The kiosk 200 b includes a camera 218 that captures an imagein which the face of the person 402 and the document 300, which includesa previously captured photograph that includes facial image 302 of theperson 402, are both visible in a captured image frame. In someembodiments, the kiosk 200 b includes one or more user input devices 230and/or output devices 232.

FIG. 6A illustrates a captured image 600 that includes a first facialimage 602 of a person 402 (FIGS. 4 and 5) and an image of a document 300that includes a second facial image 302, in accordance with someembodiments. The captured image 600 is generated by an image capturingdevice 200 such as the first image capturing device 200 a as describedwith regard to FIG. 4 or the second image capturing device 200 b asdescribed with regard to FIG. 5. In some embodiments, the captured image600 is an image frame from a video sequence. In some embodiments, thecaptured image 600 is an image frame captured as a single still image.

In FIG. 6B, a captured image 600 is shown annotated with a first regionmarker 604 indicating a portion of the captured image 600 thatcorresponds to a facial image 602 and a second region marker 606indicating a portion of the image 600 that corresponds to a facial image302 in the document 300. For example, when the captured image 600 hasbeen received by a computing system 100, region markers 604 and 606indicate portions of the image 600 that correspond to facial images 602and 302, respectively, as determined by the image analysis module 106.While the illustrative regions indicated in FIG. 6B are ovals, it willbe recognized that the portion of the image 600 that corresponds to thefacial image 602 and/or the portion of the image 600 that corresponds tothe facial image 302 in the document 300 may be rectangular regions,circular regions, polygonal regions, and/or regions that conform todetected outlines of the facial images 302 and 602. “Facial image,” asused herein, refers to a face of a person 402, a portion of a face ofthe person 402, and/or a face and other parts of the body of the person402 (e.g., the face and the person's shoulders or the person's entirebody).

FIG. 7A shows a first captured image 700 that illustrates a first facialposition 702 and FIG. 7B shows a second captured image 750 thatillustrates a second facial position 752, in accordance with someembodiments.

In FIG. 7A, the first image 700 shows the face of person 402 orientedtoward the document 300 (turned to the person's right side). When theorientation of the face of the person 402 in the captured image 700 isdifferent from the orientation of the face in second facial image 302,the ability of the image analysis module 106 to determine whether thefirst facial image matches the second facial image 302 may be impeded.In some embodiments, the image analysis module 106 is configured todetermine a first facial position 702 of the first facial image (e.g.,the facial image that has first facial position 702) and a facialposition of the second facial image 302. In some embodiments, if thefirst facial position 702 is not sufficiently similar to the secondfacial position in the second facial image 302 for image analysis module106 to determine whether matching criteria are met, the computing system100 transmits a facial position matching request to the image capturingdevice 200.

For example, in accordance with a determination by the image analysismodule 106 that a facial position adjustment is needed, the computingsystem 100 transmits to the image capturing device 200 a facial positionadjustment request, which includes a message such as “please turn yourhead to the left.” In some embodiments, in response to receiving thetransmitted request, the image capturing device 200 displays orotherwise outputs this message (e.g., via an output device 232). In someembodiments, in response to receiving the transmitted request (e.g.,subsequent to displaying the received message), image capturing device200 captures a new image 750, as shown in FIG. 7B, and sends the newimage 750 to the computing system 100. In some embodiments, thecomputing system 100 performs image analysis on the new image 750.

In some embodiments, determining whether a first facial image in a firstfacial position 702 and the second facial image meet facial positionmatching criteria includes determining whether a location of one or morefacial features (e.g., right eye, left eye, mouth, nose, and/or otheridentified facial curve or protrusion) detected in the second facialimage 302 are also detected in the first facial image in the firstfacial position 702. If the one or more facial features in the secondfacial image are not detected in the first facial position 702 of thefirst image, the computing system 100 transmits to the image capturingdevice 200 a facial position adjustment request (e.g., including amessage such as, “please turn your head to the left,” “please turn yourhead to the right,” “please tilt your head upward,” or “please tilt yourhead downward”).

In some embodiments, determining whether a first facial image in a firstfacial position 702 and a second facial image meet facial positionmatching criteria includes determining whether a face in the firstfacial position 702 is at least partially obstructed (e.g., partiallycovered by a hat) and/or determining whether a face in the second facialimage 302 is at least partially obstructed (e.g., covered by a finger).If an obstruction is detected, the computing system 100 transmits toimage capturing device 200 a facial position adjustment request (e.g.,including a message such as, “please remove your hat,” or “please moveyour finger so that it is not covering the picture of your face”).

FIGS. 8A-8B, 9A-9B, 10A-10B, 11A-11B, and 12A-12B illustrate userinterfaces and captured images associated with liveness assessments, inaccordance with some embodiments. For example a liveness assessmentassesses movement of a person that occurs in response to an instructionthat is displayed or output by a speaker. In some embodiments, aliveness assessment provides additional protection against unauthorizedaccess by ensuring that the person attempting to gain authorization(e.g., the person 402 who is presenting a document 300) is a liveindividual capable of particular movements in response to instructions.For example, the liveness assessment is used to ensure that a stillimage cannot be used to gain fraudulent access to a system. In someembodiments, the displayed instruction is randomly, pseudorandomly, orcyclically generated (e.g., so that a user must respond in real time toa prompt that is not predictable prior to the time of the accessattempt). FIGS. 8A-8B, 9A-9B, and 10A-10B illustrate use of eye-trackingto for liveness assessment, and FIGS. 11A-11B, and 12A-12B illustrateuse of language content in a message for liveness assessment.

FIGS. 8A and 9A illustrate a user interface 800 that displays a movingtarget 802 for liveness verification, in accordance with someembodiments. FIG. 8A illustrates a first state of the user interface800, which is displayed at a first time (t₀) and FIG. 9A illustrates asecond state of the user interface 800 as displayed at a second time(t₁), which is later than the first time. The user interface 800 isdisplayed by a display (e.g., an output device 232) of the imagecapturing device 200 (e.g., 200 a or 200 b). In some embodiments, themoving target 802 is an animated image, such as an animated dot. In someembodiments, the moving target 802 moves across the user interface 800(e.g., side-to-side, as shown in FIGS. 8A and 9A, vertically,diagonally, sinusoidally, and/or along another path). In someembodiments, the path of movement of the moving target 802 is apre-defined path, a randomly-generated path, a pseudorandomly generatedpath, or a path that is randomly, pseudorandomly, or cyclically selectedfrom a pre-defined set of paths. In some embodiments, the user interface800 displays a prompt (e.g., instructive text 804) to provideinstructions to a user (e.g., the person 402) for moving a facialfeature to satisfy the liveness criteria.

FIGS. 8B and 9B illustrate captured images (850, 950) that are capturedat the first time (t₀) and the second time (t₁), respectively, while theuser interface 800 is displayed. In some embodiments, the capturedimages 850 and 950 are frames of a video or still images captured by acamera 218. The captured images 850 and 950 include a first facial image852 and a second facial image 952, respectively, of the person 402. Insome embodiments, one or more facial features (e.g., one or more partsof one or both eyes 856, such as pupils, retinas, and/or irises 854) ofthe person 402 are tracked. For example, a change in the position theone or more facial features from the first image 850 to the second image950 is determined and compared to a path of movement of the movingtarget 802 displayed in the user interface 800. In this way, a person402 provides liveness verification by moving one or more facial features(e.g., changing a direction of view of the person's eyes 856) inaccordance with the path of movement of the moving target 804.

In some embodiments, to meet the movement criteria for a livenessassessment, movement of a facial feature must exceed a thresholddistance (e.g., relative to movement of a boundary of the person'sface). FIGS. 10A-10B illustrate movement of an eye (specifically, theiris 854) relative to movement of a facial image 852 of a person 402.For example, the eye movement illustrated in FIGS. 10A-10B occurs as theuser interface 800 displays a moving target 802, as illustrated in FIGS.8A and 9A. FIG. 10A illustrates the first image 850 (also shown in FIG.8B) including the first facial image 852 of the person 402 at the firsttime (t₀). A facial border 1002 corresponds to the outline of the firstfacial image 852 (e.g., as determined using an image processingtechnique, such as edge detection). FIG. 10B illustrates the secondimage 950 (also shown in FIG. 9B) including the second facial image 952of the person 402 at the second time (t₁). The face of person 402 hasmoved in the time between t₀ and t₁, by an amount illustrated by thefacial border movement distance 1004. In some embodiments, to satisfythe movement criteria for a liveness assessment, movement of the iris854 (or movement of a determined iris border 1006 that corresponds tothe iris 854), as illustrated by the iris movement distance 1008, mustexceed movement of facial border 1002 (e.g., by at least a thresholdamount).

FIGS. 11A and 12A illustrate a user interface 1100 that displayslanguage content 1102 for liveness verification, in accordance with someembodiments. FIG. 11A illustrates a first state of a user interface 1100that is displayed at a first time (t₀) and FIG. 12A illustrates a secondstate of the user interface 1100 as displayed at a second time (t₁),which is later than the first time. The user interface 1100 is displayedby a display (e.g., an output device 232) of an image capturing device200 (e.g., 200 a or 200 b). In some embodiments, the language content1102 is a word, a set of words (e.g., a phrase and/or a sentence), a setof sentences, a letter, a set of letters, a gibberish word, and/or a setof gibberish words. In some embodiments, the language content 1102 ispredetermined language content, randomly generated language content,and/or pseudorandomly generated language content. In some embodiments,the language content 1102 is cyclically, randomly, and/or pseudorandomlyselected from a set of predetermined language content items. In someembodiments, respective words in a set of words are sequentiallyhighlighted (e.g., shown with a visually distinguishing feature such asa size, font, bolding, italicizing, and/or underlining thatdistinguishes the respective word from other words) in order to indicateat a particular time that the person 402 is to read a respective wordfrom the language content 1102. In some embodiments, the user interface1100 displays or outputs by an audio output a prompt (e.g., instructivetext 1104) to provide instructions to a user (e.g., the person 402) forspeaking language content 1102 that is displayed or otherwise output.

FIGS. 11B and 12B illustrate captured images 1150 and 1250, which arecaptured at the first time (t₀) and the second time (t₁), respectively,while the user interface 1100 is displayed. In some embodiments, thecaptured images 1150 and 1250 are frames of a video or still imagescaptured by a camera 218. The captured images 1150 and 1250 includefacial images 1152 and 1252, respectively, of the person 402. In someembodiments, a position of the mouth 1154 within facial image 1152and/or a position of the mouth 1254 within the facial image 1252 isdetermined. One or more mouth shape parameters (e.g., an extent to whichthe mouth is open and/or a roundness of the mouth shape) in one or morecaptured images (e.g., 1150 and 1250) is determined and compared withone or more mouth shapes that correspond to the displayed languagecontent 1102. The person 402 provides liveness verification by speakingin response to displayed or otherwise output language content 1102. Asthe message is spoken, the person's mouth makes mouth shapes thatcorrespond to stored mouth shape information.

FIGS. 13A-13H are flow diagrams illustrating a method 1300 forauthenticating a user using facial image comparison, in accordance withsome embodiments. The method 1300 is performed at a device, such as acomputing system 100. For example, instructions for performing themethod 1300 are stored in the memory 102 and executed by theprocessor(s) 120 of the computing system 100. In FIGS. 13A-13H, dottedoutlines indicate optional operations.

In some embodiments, prior to receiving a captured image, the devicereceives (1302) an authorization request from an image capturing device200 via a first network 150. For example, the authorization request isan access request (such as a data access request, a device accessrequest, and/or a facility access request) and/or a request to conduct atransaction.

In some embodiments, in response to receiving the authorization requestfrom the image capturing device 200, the device transmits (1304), to theimage capturing device 200, a request for captured image data thatincludes a first facial image 602 and an image of a document 300 thatincludes a second facial image 302.

In some embodiments, the device determines (1306) whether the firstnetwork 150 meets image validation criteria.

In some embodiments, the first network 150 is a first financial network(e.g., a network associated with a first transaction processor) or afirst commercial network (e.g., a network associated with a particularfacility or service provider) and a second network 152 is a secondfinancial network (e.g., a network associated with a second transactionprocessor) or a second commercial network.

In some embodiments, after an authorization (e.g., for processing atransaction using a first network 150) has been granted for a particularuser (e.g., as identified by the user identification information 208transmitted from the image capturing device 200 to the computing system100), image matching determinations are no longer required forsubsequent authorizations of the user. In some embodiments, suspendingimage matching determinations lasts for a limited window of time (e.g.,five minutes or thirty minutes). In other embodiments, suspending imagematching determinations lasts indefinitely.

In some embodiments, determining whether the first network 150 meetsimage validation criteria includes (1308) accessing a stored value todetermine whether a previous authorization of a received requestoccurred. For example, in some embodiments, determining whether anauthorization has been granted for a particular user includesdetermining whether, for a previous authorization request, authorizationinformation has been received by the imaging capturing device 200 (e.g.,as described below with regard to operation 1342). If an authorizationhas not been granted for the particular user, the image validationcriteria are met (e.g., image validation is required for the currenttransaction).

In some embodiments, in accordance with a determination that the firstnetwork 150 meets the image validation criteria (e.g., image validationhas not been previously performed for a particular user), the devicetransmits (1310) the request for the captured image data that includesthe first facial image 602 and the image of the document 300 thatincludes the second facial image 302. In some embodiments, in accordancewith a determination that the first network does not meet the imagevalidation criteria (e.g., image validation has been previouslyperformed for the user, so no further image validation is required), thedevice forgoes (1312) transmitting the request for the captured imagedata that includes the first facial image 602 and the image of thedocument 300 that includes the second facial image 302.

In some embodiments, in accordance with a determination that the firstnetwork 150 does not meet the image validation criteria, the devicetransmits (1314) a request for alternative authentication data (e.g.,username, password, and/or alternative biometric data such asfingerprint scan and/or retinal scan). For example, when it isdetermined that a first network 150 is not capable of performing imagevalidation, the device requests the alternative authentication data.

In some embodiments, a second network 152 does not meet image validationcriteria (e.g., the second network 152 does not support and/or requireimage validation).

In some embodiments, in response to receiving an authorization requestfrom an image capturing device 200 via a second network 152, the deviceforgoes transmitting (1316) the request for the captured image data thatincludes the first facial image and the image of the document thatincludes the second facial image. For example, when the devicedetermines that the second network 152 is not capable of performingimage validation, the device does not request captured image data fromthe second network 152.

The device receives (1318), from an image capturing device 200, capturedimage data (e.g., an image frame 600, as illustrated in FIG. 6A) thatincludes a first facial image 602 and an image of a document 300 thatincludes a second facial image 302. The first facial image 602 and theimage of the document 300 that includes the second facial image 302 areincluded a single image frame 600.

In some embodiments, the captured image data includes (1320) an imageframe 600. In some embodiments, the captured image data includes (1322)a video stream (and the image frame 600 is a single frame of the videostream).

In some embodiments, the image of the document 300 that includes thesecond facial image 302 includes (1324) second facial image location cueinformation (e.g., the concentric rectangles indicated at 304 of FIG. 3)and determining the second portion 606 of the captured image data thatcorresponds to the second facial image 302 includes determining alocation of the facial image location cue information. Facial imagelocation cue information is described further above with regard to FIG.3. In some embodiments, the device determines the second portion 606 ofthe captured image data by determining an area bounded by the facialimage location cue information. In some embodiments, the devicedetermines the second portion 606 of the captured image data bydetermining a location of the second facial image 302 within a regionindicated by the facial image location cue information.

The device analyzes the captured image data to determine (1326) a firstportion 604 of the captured image data that corresponds to the firstfacial image 602 and a second portion 606 of the captured image datathat corresponds to the second facial image 302. For example, the imageanalysis module 106 uses edge detection techniques to determine a regionand/or outline of the first facial image 602 and the second facial image302.

In some embodiments, analyzing the captured data to determine the firstportion 604 of the captured image data that corresponds to the firstfacial image 602 includes determining (1328), in a plurality of imageframes (e.g., image frames of a video), a respective portion of arespective image frame that corresponds to the first facial image 602.In some embodiments, analyzing the captured data to determine the firstportion 604 of the captured image data that corresponds to the firstfacial image 602 includes (1330) generating the first portion 604 bycompositing a plurality of respective portions of respective imageframes from the plurality of image frames that correspond to the firstfacial image 602. For example, if a segment of the face in the firstfacial image 602 is obstructed in a first frame and a distinct segmentof the face in the first facial image 602 is obstructed in a secondframe, the obstructed segment of the face in the second frame can bereplaced with a corresponding unobstructed segment of the face from thefirst frame.

In some embodiments, analyzing the captured data to determine the firstportion 604 of the captured image data that corresponds to the firstfacial image 602 includes (1332) selecting, using the second portion 606of the captured image data that corresponds to the second facial image302, a respective portion of a respective image frame that correspondsto the first facial image 602. For example, the image analysis module106 determines one or more parameters (e.g., shape of face, location offacial features such as eyes, mouth, and nose relative to one anotherand/or relative to an outline of the face, relative sizes of facialfeatures, and/or distances between facial features) of the second facialimage 302 and uses the one or more parameters of the second facial image302 to identify corresponding parameters in the first facial image 602.

In some embodiments, analyzing the captured data to determine the secondportion 606 of the captured image data that corresponds to the secondfacial image 302 includes determining, in a plurality of image frames(e.g., image frames of a video), a respective portion of a respectiveimage frame that corresponds to the second facial image 302. In someembodiments, analyzing the captured data to determine the second portion606 of the captured image data that corresponds to the second facialimage 302 includes generating the second portion 606 by compositing aplurality of respective portions of respective image frames from theplurality of image frames that correspond to the second facial image302. For example, if a segment of the face in the second facial image302 is obstructed in a first frame and a distinct segment of the face inthe second facial image 302 is obstructed in a second frame, theobstructed segment of the face in the second frame can be replaced witha corresponding unobstructed segment of the face from the first frame.

In some embodiments, the method 1300 proceeds from operation 1326 tooptional operation 1334, as indicated at C. In some embodiments, themethod 1300 proceeds from operation 1326 to optional operation 1336, asindicated at D. In some embodiments, the method 1300 proceeds fromoperation 1326 to operation 1338, as indicated at E.

In some embodiments, after analyzing the captured image data todetermine the first portion 604 of the captured image data thatcorresponds to the first facial image 602 and the second portion 606 ofthe captured image data that corresponds to the second facial image 302,the device (1334): analyzes (1334-a) the first portion 604 of thecaptured image data to determine a first facial position 702 andanalyzes (1334-b) the second portion 606 of the captured image data todetermine a second facial position. FIG. 7A illustrates a first facialposition 702 that is different from a facial position of the secondfacial image 302. In some embodiments, the device determines (1334-c)whether facial position matching criteria are met by comparing the firstfacial position 702 with the facial position of the second facial image.

In some embodiments, in accordance with a determination that facialposition matching criteria are not met, the device transmits (1334-d),to the image capturing device 200, a facial position matching request,and the method 1300 proceeds from operation 1334-d to operation 1318, asindicated at A. Examples of facial position matching criteria and facialposition matching requests are described with regard to FIGS. 7A and 7B.

In some embodiments, in accordance with a determination that facialposition matching criteria are met, the device forgoes transmitting(1334-e), to the image capturing device 200, the facial positionmatching request.

In some embodiments, the method 1300 proceeds from operation 1334 tooptional operation 1336, as indicated at D. In some embodiments, themethod 1300 proceeds from operation 1334 to operation 1338, as indicatedat E.

In some embodiments, after analyzing the captured image data todetermine the first portion 604 of the captured image data thatcorresponds to the first facial image 602 and the second portion 606 ofthe captured image data that corresponds to the second facial image 302,the device (1336): analyzes (1336-a) the second portion 606 of thecaptured image data to determine a facial position of the second facialimage 302 and determines (1336-b) whether the second facial positionmeets facial position criteria that correspond to the document 300. Forexample, specifications for a document 300 may define facial positionrequirements, such as an orientation of a face in the facial image, adistance from the face to a border surrounding the facial image, anexpression of the face, and/or whether accessories may be worn on theface. When a facial position requirement that corresponds to a documenttype of the document 300 is not met, the facial position criteria arenot satisfied.

In accordance with a determination that the facial position criteriathat correspond to the document 300 are not met, the device transmits(1336-c), to the image capturing device 200, a facial positionadjustment request, and the method 1300 proceeds from operation 1336-cto operation 1318, as indicated at A. Examples of facial positionmatching criteria and facial position matching requests are describedwith regard to FIGS. 7A and 7B.

In accordance with a determination that the facial position criteriathat correspond to the document 300 are met, the device forgoestransmitting (1336-d), to the image capturing device 200, the facialposition adjustment request.

In some embodiments, the method 1300 proceeds from operation 1336 tooperation 1338, as indicated at E.

The device determines (1338) whether the first facial image 602 and thesecond facial image 302 meet matching criteria by comparing the firstportion 604 of the captured image data with the second portion 606 ofthe captured image data. In some embodiments, comparing the firstportion 604 of the captured image data with the second portion 606 ofthe image data includes comparing, for example, distance, size, shape,curve features, color, and/or relative properties of one or moreportions of the first image 602 and the second image 302.

In some embodiments, the image analysis module 106 determines a locationof a first facial feature (e.g., a nose) within the first portion 604 ofthe captured image data that corresponds to the first facial image 602and within the second portion 606 of the captured image data thatcorresponds to the second facial image 302. In some embodiments, theimage analysis module 106 determines a location of a second facialfeature (e.g., a left eye) within the first portion 604 of the capturedimage data and within the second portion 606 of the captured image data.A first distance between the first facial feature and the second facialfeature in the first portion 604 of the captured image data isdetermined. A second distance between the first facial feature and thesecond facial feature in the second portion 606 of the captured imagedata is determined. The first distance (e.g., relative to the size offacial image 602 in the first portion 604) is compared with the seconddistance (e.g., relative to the size of the facial image 302 in thesecond portion 606) to determine whether the first facial image and thesecond facial image meet the matching criteria.

In some embodiments, the image analysis module 106 determines a locationof a facial feature (e.g., an iris of at least one eye) within the firstportion 604 of the captured image data that corresponds to the firstfacial image 602 and within the second portion 606 of the captured imagedata that corresponds to the second facial image 302, and compares acolor of the facial feature (e.g., a color of at least one pixel) in thefirst portion 604 of the captured image data with the color of thefacial feature in the second portion 606 of the captured image data todetermine whether the first facial image 602 and the second facial image302 meet the matching criteria.

In some embodiments, the image analysis module 106 determines a shape ofa face outline within the first portion 604 of the captured image datathat corresponds to the first facial image 602 and within the secondportion 606 of the captured image data that corresponds to the secondfacial image 302, and compares the shape of the face in the firstportion 604 of the captured image data with the shape of the face in thesecond portion 606 of the captured image data to determine whether thefirst facial image 602 and the second facial image 302 meet the matchingcriteria.

In some embodiments, in lieu of and/or in addition to facial data, theimage analysis performed on the captured image data uses anotheridentifying feature of a user, such as a torso image, a whole bodyimage, a hand image, and/or a fingerprint image. For example, thecaptured image data includes a first whole body image and an image of adocument that includes a second whole body image.

In some embodiments, determining whether the first facial image 602 andthe second facial image 302 meet matching criteria includes (1340) atleast one of: comparing the first portion 604 of the captured image datawith image data that corresponds to a stored facial image or comparingthe second portion 606 of the captured image data with image data thatcorresponds to the stored facial image. For example, the stored facialimage is a reference image of the person 402 stored in the memory 102,the local authentication database 130, and/or the remote authenticationdatabase 154. The reference image is, for example, an image received bythe computing system 100 prior to an authorization request or an imagedetermined by the computing system 100 (e.g., by searching for imagesthat correspond to user information, such as the user name) in responseto receiving an authorization request.

In accordance with a determination that the first facial image 602 andthe second facial image 302 meet the matching criteria, the devicetransmits (1342) authorization information to the image capturing device200. The authorization information is, for example, transaction approvalinformation, information (e.g., a cryptographic key, a uniform resourcelocator, a passcode, and/or a parameter value) that grants access todata, to a facility, and/or to a device.

In some embodiments, in accordance with a determination that the firstfacial image 602 and the second facial image 302 do not meet thematching criteria, the device transmits (1344) authorization denialinformation to the image capturing device 200.

In some embodiments, in accordance with a determination that the firstfacial image 602 and the second facial image 302 do not meet thematching criteria, the device transmits (1346), to the image capturingdevice, a facial position adjustment request. Examples of facialposition adjustment requests are discussed above with regard to FIGS. 7Aand 7B.

In some embodiments, in lieu of receiving captured image data from animage capturing device 200 that is remote from the computing system 100,the computing system 100 captures the captured image data. For example,the computing system 100 captures the captured image data using abiometric input device 128, a camera (not shown) that is a component ofthe computing system 100, or a local camera (not shown) that is aperipheral device of the computing system 100. In this way, the samesystem that captures the image data also analyzes the image data asdescribed with regard to FIGS. 13C-13E. For example, in someembodiments, a kiosk (e.g., similar to the kiosk 200 b as illustrated inFIG. 5) includes a computing system 100 with a camera that captures animage of the person 402 and the document 300.

In some embodiments, the captured image data includes (1348) a thirdfacial image (e.g., the second facial image 952 in the second image 950)captured at a first time (e.g., t₁) that is different from a second time(e.g., to or other time) at which at least one other facial image (e.g.,a facial image of the person 402 in one or more of the images 600, 700,750, or 850), is captured. In some embodiments, the device determinesthat a facial feature (e.g., the iris 854) of the third facial image anda corresponding facial feature of the at least one other facial imagemeet movement criteria by comparing the facial feature of the thirdfacial image with the corresponding facial feature of the at least oneother facial image. For example, a position of the iris 854 in the firstimage 850 is compared with a position of the iris 854 in the secondimage 950, as illustrated by the distance 1008 in FIGS. 10A-10B. Inaccordance with a determination that the facial feature of the thirdfacial image and the corresponding facial feature of the at least oneother facial image meet the movement criteria, the device includes, inthe transmitted authorization information, an indication that themovement criteria are met. In some embodiments, the transmittedauthorization information includes multiple transmissions (e.g., a firsttransmission, which is transmitted in accordance with a determinationthat the first facial image and the second facial image meet thematching criteria and a second transmission, which is distinct from thefirst transmission, and transmitted in accordance with a determinationthat the facial feature of the third facial image and the correspondingfacial feature of the at least one other facial image meet the movementcriteria). In some embodiments, the authorization information istransmitted in a single transmission. In some embodiments, in accordancewith a determination that the facial feature of the third facial imageand the corresponding facial feature of the at least one other facialimage do not meet the movement criteria, the device transmitsauthorization denial information to the image capturing device.

In some embodiments, determining that the facial feature (e.g., the iris854) of the third facial image (e.g., the image 952) and thecorresponding facial feature of the at least one other facial image(e.g., the image 852) meet the movement criteria includes (1350)determining a facial feature movement path (e.g., as illustrated by themovement of the iris 854 in FIGS. 10A-10B) by the comparing of thefacial feature of the third facial image with the corresponding facialfeature of the at least one other facial image and determining that alength of the facial feature movement path meets threshold distancecriteria.

In some embodiments, determining that the facial feature (e.g., the iris854) of the third facial image (e.g., the image 952) and thecorresponding facial feature of the at least one other facial image(e.g., the image 852) meet the movement criteria includes (1352)transmitting, to the image capturing device 200, information fordisplaying (e.g., on the display of the imaging capturing device 200,such as the output device 232), a target (e.g., a moving target 802),which moves along a target path (e.g., as illustrated by the userinterface 800 as shown in FIG. 8A and FIG. 9A). A facial featuremovement path is determined by comparing the facial feature of the thirdfacial image with the corresponding facial feature of the at least oneother facial image (e.g., as illustrated by movement of the iris 854 inFIGS. 10A-10B). The target path is compared with the facial featuremovement path to determine whether a path matching value meets pathsimilarity criteria. In some embodiments, in accordance with adetermination that the path matching value meets the path similaritycriteria, the movement criteria are met.

In some embodiments, determining that the facial feature (e.g., the iris854) of the third facial image (e.g., the image 952) and thecorresponding facial feature of the at least one other facial image(e.g., the image 852) meet the movement criteria includes (1356)determining a facial feature movement path by comparing the facialfeature of the third facial image with the corresponding facial featureof the at least one other facial image (e.g., as illustrated by movementof the iris 854 in FIGS. 10A-10B). A facial boundary movement path isdetermined by comparing a facial boundary (e.g., the boundary 1002) ofthe third facial image (e.g., the image 952) with a corresponding facialboundary (e.g., the boundary 1002) of the at least one other facialimage (e.g., the image 852). In some embodiments, in accordance with adetermination that a length of the facial feature movement path (e.g.,the path 1008) exceeds a length of the facial boundary movement path(e.g., the path 1004) by more than a threshold distance, the movementcriteria are met.

In some embodiments, the facial feature of the third image includes(1358) at least a part (e.g., the iris 854) of one or more eyes 856 in aface.

In some embodiments, the device transmits (1360), to the image capturingdevice 200, a message (e.g., language content 1102, as shown in FIGS.11A and 12A). For example, the message is configured for output by anoutput device 232 of the image capturing device 200, such as a displayand/or a speaker. In some embodiments, the message is randomly orpseudorandomly generated (1362). In some embodiments, the message israndomly or pseudorandomly selected (1364) from a predefined set ofmessages. In some embodiments, in addition to and/or in lieu ofdisplaying the language content 1102, the device generates an audiooutput of a message with the language content 1102.

The device receives (1366), from the image capturing device 200, arecording (e.g., a recording captured by the camera 218 and/or amicrophone of the input device 230) of the user speaking the message.

In some embodiments, the recording is (1370) an audio recording (e.g.,recorded using a microphone of the input device 230 of the imagecapturing device 200). In some embodiments, determining whether therecording meets message speaking criteria includes (1370) determiningwhether at least a portion of the audio recording corresponds to apredetermined audio profile for the message. For example, an audiomatching algorithm determines whether an audio recording matches arecorded or generated audio file stored by the liveness analysis module119 (e.g., within a predetermined tolerance).

In some embodiments, the recording is (1372) a video recording (e.g.,recorded by the camera 218 of the image capturing device 200), whichcaptures movements of the user's face while the user reads the message(e.g., a video recording that includes video frames as described withregard to FIGS. 11B and 12B). In some embodiments, determining whetherthe recording meets message speaking criteria includes (1372)determining whether at least a portion of the video recording (e.g.,mouth shape parameters of the mouth 1254 in one or more frames of thevideo, as described with regard to FIGS. 11B and 12B) corresponds to aset of predefined facial movements for the message (e.g., video and/orimages stored by the liveness analysis module 119). For example, theimage analysis module 106 compares mouth shapes in the stored videoand/or images with mouth shapes in the received recording (e.g., withina predetermined tolerance).

In some embodiments, the recording is a set of still images (e.g.,captured by the camera 218 of image capturing device 200), which capturemovements of the user's face while the user reads the message. In someembodiments, determining whether the recording meets message speakingcriteria includes determining whether mouth shape parameters of themouth 1254 in at least a subset of the set of still images correspondsto a set of predefined facial movements for the message. For example,the image analysis module 106 compares a shape of a mouth in one or morestored images with the shape of the mouth in one or more images from thereceived set of still images (e.g., within a predetermined tolerance).

In some embodiments, the device determines (1368) whether the recordingmeets message speaking criteria, and in accordance with a determinationthat the recording meets message speaking criteria, the device includes(1374), in the transmitted authorization information, an indication thatthe message speaking criteria are met. In some embodiments, thetransmitted authorization information includes multiple transmissions(e.g., a first transmission, which is transmitted in accordance with adetermination that the first facial image and the second facial imagemeet the matching criteria, and a second transmission, which is distinctfrom the first transmission. The second transmission is transmitted inaccordance with a determination that the recording meets messagespeaking criteria). In some embodiments, the authorization informationis transmitted in a single transmission. In some embodiments, inaccordance with a determination that the recording does not meet messagespeaking criteria, the device transmits authorization denial informationto the image capturing device.

Features of the present invention can be implemented in, using, or withthe assistance of a computer program product, such as a storage medium(media) or computer readable storage medium (media) having instructionsstored thereon/in which can be used to program a processing system toperform any of the features presented herein. The storage medium (e.g.,the memory 102 and the memory 202) can include, but is not limited to,high-speed random access memory, such as DRAM, SRAM, DDR RAM or otherrandom access solid state memory devices, and may include non-volatilememory, such as one or more magnetic disk storage devices, optical diskstorage devices, flash memory devices, or other non-volatile solid statestorage devices. In some embodiments, the memory 102 and the memory 202include one or more storage devices remotely located from the CPU(s) 120and 220. The memory 102 and the memory 202, or alternatively thenon-volatile memory device(s) within these memories, comprises anon-transitory computer readable storage medium.

Communication systems as referred to herein (e.g., the communicationsystem 124 and the communication system 226) optionally communicate viawired and/or wireless communication connections. Communication systemsoptionally communicate with networks (e.g., the networks 150 and 152),such as the Internet, also referred to as the World Wide Web (WWW), anintranet and/or a wireless network, such as a cellular telephonenetwork, a wireless local area network (LAN) and/or a metropolitan areanetwork (MAN), and other devices by wireless communication. Wirelesscommunication connections optionally use any of a plurality ofcommunications standards, protocols and technologies, including but notlimited to Global System for Mobile Communications (GSM), Enhanced DataGSM Environment (EDGE), high-speed downlink packet access (HSDPA),high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO),HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), nearfield communication (NFC), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 102.11a,IEEE 102.11ac, IEEE 102.11ax, IEEE 102.11b, IEEE 102.11g and/or IEEE102.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol fore-mail (e.g., Internet message access protocol (IMAP) and/or post officeprotocol (POP)), instant messaging (e.g., extensible messaging andpresence protocol (XMPP), Session Initiation Protocol for InstantMessaging and Presence Leveraging Extensions (SIMPLE), Instant Messagingand Presence Service (IMPS)), and/or Short Message Service (SMS), or anyother suitable communication protocol, including communication protocolsnot yet developed as of the filing date of this document.

It will be understood that, although the terms “first,” “second,” etc.may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the claims. Asused in the description of the embodiments and the appended claims, thesingular forms “a,” “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willalso be understood that the term “and/or” as used herein refers to andencompasses any and all possible combinations of one or more of theassociated listed items. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, steps, operations, elements, components, and/orgroups thereof.

As used herein, the term “if” may be construed to mean “when” or “upon”or “in response to determining” or “in accordance with a determination”or “in response to detecting,” that a stated condition precedent istrue, depending on the context. Similarly, the phrase “if it isdetermined [that a stated condition precedent is true]” or “if [a statedcondition precedent is true]” or “when [a stated condition precedent istrue]” may be construed to mean “upon determining” or “in response todetermining” or “in accordance with a determination” or “upon detecting”or “in response to detecting” that the stated condition precedent istrue, depending on the context.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the claims to the precise forms disclosed. Many modifications andvariations are possible in view of the above teachings. The embodimentswere chosen and described in order to best explain principles ofoperation and practical applications, to thereby enable others skilledin the art.

1. (canceled)
 2. A computer-implemented method, comprising: at acomputing system including one or more processors and memory storing oneor more programs configured for execution by the one or more processors:receiving, from a camera on an electronic device distinct from thecomputing system, a single captured image of a user holding anidentification document, wherein the single captured image includes theuser's live face and a photograph of the user's face in theidentification document, and determining whether the user's live faceand the photograph of the user's face in the identification documentmeet matching criteria; and in accordance with a determination that theuser's live face and the photograph of the user's face in theidentification document meet the matching criteria, transmittingauthorization information for the user to the electronic device.
 3. Themethod of claim 2, further comprising, in accordance with adetermination that the user's live face and the photograph of the user'sface in the identification document do not meet the matching criteria,transmitting, to the electronic device, a facial position adjustmentrequest.
 4. The method of claim 2, wherein the single captured image isa frame in a video stream.
 5. The method of claim 4, further includingdetermining whether the user's live face meets liveness criteria,wherein the authorization information for the user is transmitted to theelectronic device further in accordance with a determination that theuser's live face meets the liveness criteria.
 6. The method of claim 4,further comprising: analyzing the video stream to determine: a firstportion of the single captured image that corresponds to the user's liveface; and a second portion of the video stream that corresponds to thephotograph of the user's face in the identification document.
 7. Themethod of claim 6, further including generating the first portion of thevideo stream by compositing a plurality of respective portions ofrespective image frames from the video stream that correspond to theuser's live face.
 8. The method of claim 6, wherein analyzing the videostream to determine the first portion of the video stream thatcorresponds to the user's live face includes selecting, using the secondportion of the video stream that corresponds to the photograph of theuser's face in the identification document, a respective portion of arespective image frame that corresponds to the user's live face.
 9. Themethod of claim 6, including: after analyzing the video stream todetermine the first portion of the video stream that corresponds to theuser's live face and the second portion of the video stream thatcorresponds to the photograph of the user's face in the identificationdocument: analyzing the first portion of the video stream to determine afirst facial position; analyzing the second portion of the video streamto determine a second facial position; determining whether facialposition matching criteria are met by comparing the first facialposition with the second facial position; and in accordance with adetermination that facial position matching criteria are not met,transmitting, to the electronic device, a facial position matchingrequest.
 10. The method of claim 2, wherein determining whether theuser's live face and the photograph of the user's face in theidentification document meet matching criteria includes at least one of:comparing the user's live face with a stored facial image of the user;and comparing the photograph of the user's face with the stored facialimage.
 11. The method of claim 2, including, in accordance with adetermination that the user's live face and the photograph of the user'sface in the identification document do not meet the matching criteria,transmitting authorization denial information to the electronic device.12. The method of claim 2, including: prior to receiving the singlecaptured image: receiving an authorization request from the electronicdevice via a first network; determining whether the first network meetsimage validation criteria; and in accordance with a determination thatthe first network meets the image validation criteria, transmitting arequest to the electronic device for the single captured image.
 13. Themethod of claim 12, wherein determining whether the first network meetsimage validation criteria includes accessing a stored value to determinewhether a previous authorization of a received request occurred.
 14. Themethod of claim 12, including, in accordance with a determination thatthe first network does not meet the image validation criteria,transmitting a request for alternative authentication data.
 15. Themethod of claim 12, including, receiving an authorization request fromthe electronic device via a second network; and in response to receivingthe authorization request from the electronic device via the secondnetwork, forgoing transmitting the request for the single capturedimage.
 16. A computing system, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and are configured for execution by the one or moreprocessors, the one or more programs including instructions for:receiving, from a camera on an electronic device distinct from thecomputing system, a single captured image of a user holding anidentification document, wherein the single captured image includes theuser's live face and a photograph of the user's face in theidentification document, and determining whether the user's live faceand the photograph of the user's face in the identification documentmeet matching criteria; and in accordance with a determination that theuser's live face and the photograph of the user's face in theidentification document meet the matching criteria, transmittingauthorization information for the user to the electronic device.
 17. Anon-transitory computer readable storage medium storing one or moreprograms, the one or more programs comprising instructions, which whenexecuted, cause a computing system to: receive, from a camera on anelectronic device distinct from the computing system, a single capturedimage of a user holding an identification document, wherein the singlecaptured image includes the user's live face and a photograph of theuser's face in the identification document, and determine whether theuser's live face and the photograph of the user's face in theidentification document meet matching criteria; and in accordance with adetermination that the user's live face and the photograph of the user'sface in the identification document meet the matching criteria, transmitauthorization information for the user to the electronic device.